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Automated Test Execution Framework with Central Man- 
agement 

CROSS-REFERENCE TO RELATED APPLICATIONS 
5 [0001] This application claims the benefit of Provi- 

sional Application No. 60/443,794, entitled "Automated Test 
Execution Framework with Central Management" . 

[0002] This application is related to Provisional Ap- 

plication No. 60/443,795, entitled "Parallel Test Execution on 
10 Low-End Emulators and Devices" . 

BACKGROUND OF THE INVENTION 

1. Field of the Invention, 

[0003] This invention relates to improvements in soft- 

ware and hardware design verification. More particularly, this 
15 invention relates to methods and systems for centrally managing 
the execution of multiple test suites on different platforms 
for verification of different hardware and software. 

2. Description of the Related Art, 

[0004] The meanings of acronyms and certain terminology 

20 used herein are given in Table 1. The terms Sun, Sun Microsys- 
tems, Java, J2EE, J2ME, J2SE, and the Sun logo are trademarks 
or registered trademarks of Sun Microsystems, Inc., in the 
United States of America and other countries. All other company 
and product names may be trademarks of their respective compa- 
25 nies. A portion of the disclosure of this patent document con- 
tains material that is subject to copyright protection. The 
copyright owner has no objection to the facsimile reproduction 
by anyone of the patent document or the patent disclosure, as 
it appears in the Patent and Trademark Office patent file or 
30 records, but otherwise reserves all copyright rights whatso- 
ever. 
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Table 1 



API 


Application programming interface 


CLDC 


Connected, limited device configuration. CLDC is 
suitable for devices with 16/32-bit RISC/CISC 
microprocessors/controllers, having as little as 
160 KB of total memory available. 


HTML 


Hypertext markup language 


J2EE 


Java 2 Enterprise Edition 


J2ME 


Java 2 Micro Edition 


J2SE 


Java 2 Standard Edition 


JAD 


Java application descriptor 


JAR 


Java archive 


MIDlet 


A MIDP application 


MIDP 


Mobile information device profile. A set of Java 
APIs, which, together with the CLDC, provides a 
complete J2ME application runtime environment 
targeted at mobile information devices. 


XML 


Extensible markup language 



[0005] Tools have been developed in recent years to aid 

in the design verification of hardware and software systems, 
5 for example software suites, hardware circuitry, and programma- 
ble logic designs. In order to assure that the design complies 
with its specifications, it is common to generate a large num- 
ber of input or instruction sequences to assure that the design 
operates as intended under a wide variety of circumstances. In 

10 general, test systems produce a report indicating whether tests 
have been passed or failed, and, in some cases may even indi- 
cate a module that is estimated to be faulty. 

[0006] Conventionally, test systems employing complex 

test suites employ a computer- implemented testing framework for 

15 computing devices, such as mobile information devices, and for 
software designed to run on such devices. A developer submits a 
computing product under development, typically a computing de- 
vice or software that is designed to run on the device to the 
test system, which runs a selected battery of test programs on 

2 0 the product while monitoring its behavior. Each product under 
test requires an instance of an execution test harness, or the 
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use of a stand-alone test execution API . The latter is de- 
scribed in commonly assigned Application No. 10/347,748, enti- 
tled "Generating Standalone MIDlets from a Testing Harness" , 
which is herein incorporated by reference. 
5 [0007] In environments where testing of a product is 

ongoing, different aspects may be tested by different teams. As 
test results are evaluated, it is often necessary to revise the 
product under test, or to modify the test suites themselves. In 
such an environment, communicating such revisions to the dif- 

10 ferent testing teams, maintaining version control, synchroniza- 
tion among the teams, and generally coordinating the testing 
activities is a difficult management problem. Errors could re- 
sult in inappropriate testing and the wastage of valuable time 
and testing resources. Indeed, failure of coordination could 

15 result in the release of an inadequately tested product into 
the marketplace. A related problem when many test suites are 
being concurrently executed is the effort of setting up each 
test suite with its own test harness and environment. Bundling 
the test harness with the test suite is not a good solution, as 

20 the effort in maintaining up-to-date versions becomes formida- 
ble as the number of concurrently operating test suites in- 
creases, and when the product or the test suites are frequently 
modified by different test teams. 

SUMMARY OF THE INVENTION 
25 [0008] In accordance with a disclosed embodiment of the 

invention, a test execution system or framework consists of one 

central repository and includes a platform editor (also known 

as an administrative tool) , which functions as a management 

unit. The central repository contains available test suites and 

30 binary files that constitute a single test execution harness. 

Using the management unit, a system administrator is enabled to 

control active versions of the various test suites, and their 

individual configurations. End users of this system install 
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clients of the central repository, using a system-provided in- 
staller program. At the client, an execution script and a local 
configuration file are created. When the execution script is 
executed by the client, the binary files at the central reposi- 
5 tory are accessed, causing the test harness program to load at 
the client with all designated test suites being installed, 
configured and ready for execution. The client always has the 
most updated versions of all test suites, as configured by the 
system administrator, using the management unit. It is an ad- 

10 vantage of the system that all necessary information is ob- 
tained from a single central location, without need for dis- 
tributing local copies of the test harness or test suites, 
which could lead to loss of synchronization. 

[0009] The invention provides a method for testing a 

15 plurality of computing products, which is carried out by pro- 
viding a central repository holding data structures that in- 
clude platforms, test suites, an execution test harness, and an 
installer. The method is further carried out by downloading and 
installing selected ones of the platforms and test suites from 

20 the central repository to a plurality of clients of the central 
repository, executing the execution test harness using the bi- 
nary files residing on the central repository, and providing a 
platform editor for modifying at least one of the platforms, 
the test suites, and the execution test harness. 

25 [0010] According to one aspect of the method, different 

selected platforms and test suites are installed on different 
clients . 

[0011] According to another aspect of the method, the 

different clients execute the test suites concurrently. 
30 [0012] According to a further aspect of the method, the 

different clients execute the test suites at different times. 

[0013] The invention provides a computer software prod- 

uct, including a computer-readable medium in which computer 
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program instructions are stored, which instructions, when read 
by a computer, cause the computer to perform a method for test- 
ing a plurality of computing products, which is carried out by 
defining a central repository holding data structures that in- 
5 elude platforms, test suites, an execution test harness, and an 
installer. The method is further carried out by downloading se- 
lected ones of the platforms and the test suites at a plurality 
of clients of the central repository, each of which can execute 
the execution test harness using the binaries residing on the 
10 central repository, and defining a platform editor for modify- 
ing any of the platforms, the test suites, and the execution 
test harness . 

[0014] The invention provides a test execution system 

for testing a plurality of computing products, including a cen- 

15 tral repository holding data structures, the data structures 
including platforms, test suites, an execution test harness, 
and an installer for downloading and installing selected plat- 
forms and test suites at a plurality of clients of the central 
repository, and a platform editor for modifying any of the 

20 platforms, the test suites, and the execution test harness. The 
execution test harness is run by the clients using the binaries 
residing on the central repository. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0015] For a better understanding of the present inven- 

25 tion, reference is made to the detailed description of the in- 
vention, by way of example, which is to be read in conjunction 
with the following drawings, wherein like elements are given 
like reference numerals, and wherein: 

[0016] Fig. 1 is a block diagram of a system for cen- 

30 trally managing the simultaneous or sequential execution of 
multiple test suites on different platforms for verification of 
different hardware and software in accordance with a disclosed 
embodiment of the invention; 
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[0017] Fig. 2 is a functional block diagram of a ses- 

sion implementation in the system shown in Fig. 1, in accor- 
dance with a disclosed embodiment of the invention; 

[0018] Fig. 3 is a detailed block diagram illustrating 

5 the modification of records in the session implementation of 
Fig. 2, in accordance with a disclosed embodiment of the inven- 
tion; 

[0019] Fig. 4 is a flow chart illustrating aspects of a 

method of conflict resolution in accordance with a disclosed 
10 embodiment of the invention; and 

[0020] Fig. 5 and Figs. 5AA-5AJ, 5BA-5BJ, 5CA-5CJ, 5DC- 

5DJ, 5EC-5EH, 5FC-5FH, AND 5GG-5GH, collectively referred to 
herein as Fig. 5, are respectively a small scale view of a 
class diagram of the system shown in Fig. 1, and partial views 
15 thereof, wherein the small scale view indicates the positions 
of the parts shown in the partial views. 

DETAILED DESCRIPTION OF THE INVENTION 

[0021] In the following description, numerous specific 

details are set forth in order to provide a thorough under- 
2 0 standing of the present invention. It will be apparent to one 
skilled in the art, however, that the present invention may be 
practiced without these specific details. In other instances 
well-known circuits, control logic, and the details of computer 
program instructions for conventional algorithms and processes 
25 have not been shown in detail in order not to unnecessarily ob- 
scure the present invention. 

[0022] Software programming code, which embodies as- 

pects of the present invention, is typically maintained in per- 
manent storage, such as a computer readable medium. In a cli- 
30 ent/server environment, such software programming code may be 
stored on a client or a server. The software programming code 
may be embodied on any of a variety of known media for use with 
a data processing system, This includes, but is not limited to, 
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magnetic and optical storage devices such as disk drives, mag- 
netic tape, compact discs (CD's), digital video discs (DVD's), 
and computer instruction signals embodied in a transmission me- 
dium with or without a carrier wave upon which the signals are 
5 modulated. For example, the transmission medium may include a 
communications network, such as the Internet. 

System Architecture. 

[0023] Turning now to the drawings, reference is ini- 

tially made to Fig. 1, which is a high level block diagram of a 

10 system 10 for centrally managing the simultaneous or sequential 
execution of multiple test suites on different platforms for 
verification of different hardware and software in accordance 
with a disclosed embodiment of the invention. The heart of the 
system 10 is a central repository 12, which can reside on a lo- 

15 cal or a remote server, and which contains data structures nec- 
essary for multiple clients (test consoles) of the central re- 
pository 12 to perform testing simultaneously, or at different 
times. The central repository 12 holds data structures that de- 
fine platforms 14, test suites 16, an execution test har- 

20 ness 18, and an installer 20. The installer 20 creates a script 
for launching the framework. The script includes paths to bi- 
nary files of the test execution framework. The binaries them- 
selves are located in only one place, the central reposi- 
tory 12. Centrally locating the binary files is highly advanta- 

2 5 geous, as only one instance of each binary file need be up- 

dated. Furthermore, each user is guaranteed to see the most 
current version of the framework. Because distribution of local 
copies of the binaries is avoided, users need not be concerned 
about having outdated software. The execution test harness 18 

3 0 may be implemented as a modification of the "Java Device Test 

Suite" execution framework (JDTS) (version 1.0 or higher), 
available from Sun Microsystems, Inc., which employs MIDP. 
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[0024] Typically the installation is packaged in an ar- 

chive, such as a JAR file. The system 10 is capable of being 
controlled by a single operator 22, using a client terminal 24. 
However, in many environments it is desirable that a superuser, 
5 or administrator 26, manage the central repository 12. This is 
done using a platform editor 28, which acts as a management 
unit. Using the platform editor 28, the administrator 26 is ca- 
pable of reconfiguring the platforms 14, test suites 16, execu- 
tion test harness 18, and installer 20. An end user 30 inter- 

10 acts with the central repository 12 via a client terminal 32. 
The end user 3 0 launches an execution package, which is first 
downloaded and installed from the central repository 12 . The 
end user 3 0 and the terminal 32 may be co- located with the 
other components of the system 10, or alternatively may be lo- 

15 cated remotely and connected to the central repository 12 via a 
data network. Although only one instance of the end user 3 0 is 
shown, it will be understood that many end users can interact 
with the central repository 12 concurrently, or at different 
times, using the same or different platforms and the same or 

20 different test suites. The end user 30 and the terminal 32 are 
referred to hereinbelow as a client 34. 

[0025] The system 10 is suitable for use in many devel- 

opment environments, including MIDlet development using J2ME. 
It may be adapted to J2EE and J2SE environments using program- 

25 ming techniques known to the art. 

System - Functional Organization. 

[0026] Continuing to refer to Fig. 1, the central re- 

pository 12 contains test parameters, platform configuration 
parameters, framework parameters and the tests themselves. The 
30 end user 30 makes use of the central repository 12 in a "ses- 
sion" . In an implementation of a session for a framework appli- 
cation in the execution state, contents of the central reposi- 
tory 12 are stored and loaded. The disclosure of the system im- 
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plementation is common to the platform editor 28, which manages 
platforms by using the central repository 12, and by the execu- 
tion framework, including the installer 20, which creates an 
execution script and local configuration files. The execution 
5 test harness 18 is not downloaded. Its binary files remain on 
the central repository 12 . 

[0027] Reference is now made to Fig. 2, which is a high 

level functional block diagram of an implementation of the 
platform editor 28 (Fig. 1) in accordance with a disclosed em- 

10 bodiment of the invention. A class Platf ormSessionManager 3 6 
has methods for loading and saving platforms and sessions. 

[0028] A class Platform 38 encapsulates all platform spe- 

cific information. An instance of the class Platform 38 in- 
cludes such information as available test suites, and their re- 

15 spective properties, as well as other platform-specific infor- 
mation. The test suites and their properties are managed by a 
class TestSuitesManager 40. Platform-specific information is 
managed by a class EnvironmentView 42. Tests in the test suites 
can be excluded from performance by a class ExcludedTests- 

20 View 44 . 

[0029] A class Session 46 encapsulates all information 

specified by the client 34 (Fig. 1) . Such client specific in- 
formation includes the basic properties of the test suites and 
the platform that applies to a particular session of the cli- 
25 ent 34. 

[0030] The class Platform 38 and the class Session 46 

interact with the class TestSuitesManager 40, the class Envi- 
ronmentView 42, and the class ExcludedTestsView 44. 

[0031] The class TestSuitesManager 40 interacts with a 

30 table TestSuiteTable 48, which contains records of the differ- 
ent test suites, as indicated by a class TestSuiteTable 50. 

[0032] The class EnvironmentView 42 interacts with a 

table Environment Table 52, which contains records of known 
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platform environments, as indicated by a representative table 
Environment Record 54 . 

[0033] The class ExcludedTestsView 44 interacts with a 

table ExcludedTestsTable 56, which contains records of excluded 
5 tests, as indicated by a representative class ExcludedTestsRe- 
cord 58 . 

[0034] Records originating from the class 

TestSuiteTable 50, the table Environment Record 54, and the 
class ExcludedTestsRecord 58, are included in a class re- 
10 cord 60, and initially written to an interface table 62, before 
being ultimately transferred to an appropriate one of the table 
Environment Table 52, the table EnvironmentRecord 54, or the ta- 
ble ExcludedTestsTable 56. The class record 60 is detailed in 
Listing 1 . 

15 [0035] A class XMLAccessManager 64 accepts requests 

from the class TestSuitesManager 40, the class Environment - 
View 42, and the class ExcludedTestsView 44 for read or write 
operations to and from the class Platform 38 and the class Ses- 
sion 46. The class XMLAccessManager 64 manages queues of read 

2 0 requests 66, write requests 68, and executes them sequentially. 

[0036] A class XMLAccess 70 contains a single access 

point to XML files that represent platforms and sessions. It 
has methods for read/write access to these XML files. 

System Operation - Platform Editor. 

25 [0037] Referring again to Fig. 1 and Fig. 2, a typical 

platform using the platform editor 2 8 is shown by the pseudo- 
code fragment of Listing 2 . The output of the procedure shown 
in Listing 2 is a XML file representing a platform, appropri- 
ately configured and accessible to the end user 30 (Fig. 1) . 

30 The configuration parameters of platforms and test suites are 
written such that they contain variables, which can be replaced 
by the end user 30. 
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[0038] Reference is now made to Fig. 3, which is a 

block diagram illustrating the modification of records in the 
class record 60 (Fig. 2) using the procedure of Listing 2, in 
which framework components 72, 74 may be extracted from the ta- 
5 ble TestSuiteTable 50, the table Environment Table 52, and the 
table ExcludedTestsTable 56. 

[0039] For example, assume that a test suite x has a 

parameter that points to its classes directory. This parameter 
should be written as follows: 
10 TestClassesDir = PARENT_DIR/bin/classes . 

[0040] A system component replaces the parameter PAR- 

ENT_DIR according to an entry in a local configuration file, 
local . configuration, which contains the path to the directory 
TestClassesDir. This operation is implemented using Listing 3, 
15 which is a pseudo-code fragment from the class record 60 
(Fig. 2) . 

System Operation - Client. 

[0041] With continued reference to Fig. 1 and Fig. 2, 

after the client 34 loads a platform from the central reposi- 

20 tory 12, it creates a session object, that is an instance of 
the class Session 46, which is based on the platform object, 
that is the class Platform 38. This session object contains 
only those basic properties that can be modified by the cli- 
ent 34. The session and platform objects contain all the infor- 

25 mation taken from the central repository 12, using the path re- 
placement mechanism disclosed above. The client 34 can then op- 
erate as if a conventional local installation of the test suite 
and harness had occurred. The sequence of events is illustrated 
in the pseudocode of Listing 4. 

3 0 Conflict Resolution - General. 

[0042] Continuing to refer to Fig. 1 and Fig. 2, as 

noted above, the central repository 12 (Fig. 1) contains all 
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available platform and test suite properties, which can be ed- 
ited by the administrator 26. These properties are logically 
divided into basic and advanced categories. Basic properties 
are accessible and locally modifiable by the client 34 
5 (Fig. 1), while advanced properties are generally read-only as 
to the client 34. The end user 30 only has read-only access to 
the central repository 12. Thus, if a particular property were 
changed by the administrator 26 and written to the central re- 
pository 12, and were also changed locally by the end user 30, 

10 conflicts and loss of data coherence could easily arise at that 
particular client . 

[0043] Issues concerning shared properties are dealt 

with according to a system- imposed policy. The implementation 
is based on the system component disclosed above. It will be 

15 recalled that the client 34 creates a session object, which is 
based on the platform object that it received from the central 
repository 12. The session object contains the same basic pa- 
rameters as the platform object. As also noted above, the plat- 
form and session objects consist of test suites, framework and 

20 platform-specific environment properties. Each test suite and 
property is represented by the class record 60, shown in List- 
ing 1, and its extensions, the class TestSuiteTable 50, the 
class Environment Record 54, and the class ExcludedTestsRe- 
cord 58. 

25 [0044] The fields of the class record 60 on which the 

resolve mechanism is based on are: 

public static final int COUNTER=10; and 
public static final int M0DIFIED=11. 
[0045] The resolution mechanism has a server-side com- 

30 ponent, implemented in the central repository 12 (Fig. 1), and 
a client-side component involving the client 34. 

[0046] Reference is now made to Fig. 4, which is a flow 

chart illustrating aspects of a method of conflict resolution 
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in accordance with a disclosed embodiment of the invention. The 
process illustrated in Fig. 4 is disclosed with further refer- 
ence to Fig. 1 and Fig. 2. 

[0047] At initial step 76, the administrator 2 6 opens a 

5 platform of the central repository 12, using the platform edi- 
tor 28, using a method. 

[0048] Next, at step 78, a backup copy of the platform 

(class Platform 38) . A copy of the test suites manager (class 
TestSuitesManager 40) is prepared, which is useful for compar- 
10 ing the values of properties before and after their changes, 
and for updating the COUNTER and MODIFIED fields of the class 
record 6 0 . 

[0049] Next, at step 80, properties are changed by the 

administrator 26. 

15 [0050] Next, at step 82, the modified platform, and the 

backup copy of the test suites manager are saved to the central 
repository 12 . It will be noted that any changes made to the 
class TestSuitesManager 40 properties during step 80 are now 
effectively nullified. The save methods invoked in step 82 also 

20 updates the COUNTER and the MODIFIED fields of the class re- 
cord 60, by comparing values of the modified record before and 
after the changes. Properties held in the tables of the table 
TestSuiteTable 50 and the table Environment Table 52 are saved 
after they are resolved by a class RecordAdder, which is shown 

25 in further detail in Listing 7 and Listing 8. The class Recor- 
dAdder compares values of the properties and adds them to ta- 
bles of the table TestSuiteTable 50 and the table Environment- 
Table 52 after the process of updating and conflict resolution 
has completed. The values are then saved into a session XML 

30 file. 

[0051] The process then ends at final step 84. Further 

details of the implementation of the process shown in Fig. 4 
are given in Listing 6. 
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[0052] The client 34 may also make property modifica- 

tions, using the method disclosed with reference to Fig. 4, ex- 
cept that the modifications are made on the session object 
(class Session 46), instead of the platform object. The details 
5 are not repeated in the interest of brevity. 

Conflict Resolution - Automatic and Manual Aspects. 

[0053] Assuming that some properties at the client and 

the server were changed as described above, in order to deter- 
mine the presence of conflicts and to resolve them, the proce- 

10 dure shown in pseudocode in Listing 7 is performed. As shown in 
Listing 8, implementation of a class RecordAdder is actually 
the core of the mechanism. 

[0054] If it is determined from the operations con- 

ducted according to Listing 7 and Listing 8 that some property 

15 is in conflict, then the system provides a Wizard to the user 
that displays the conflicting property and its values. The user 
is invited to select one of the conflicting values. The se- 
lected value is added to a table of the class TestSuiteTable 50 
or the table Environment Table 52 . The default choice is the 

2 0 platform-provided value. 

Implementation . 

[0055] Reference is now made to Fig. 5, which is a de- 

tailed class diagram implementing the system 10 (Fig. 1) , in 
accordance with a disclosed embodiment of the invention. Fig. 5 
25 generally includes the classes shown in Fig. 2 in further de- 
tail as well as derived classes and interfaces. The same 
classes in Fig. 5 and Fig. 2 are sometimes referenced by dif- 
ferent reference numerals in Fig. 5, in order to disclose vari- 
ants and subclasses more clearly. 

3 0 Table. 

[0056] A general access to tables is provided by a ta- 

ble access interface TableDataAccess 90, which includes several 
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methods for managing table contents. A method add 92 adds a re- 
cord to a specified table. The added record is an instance of 
an abstract record class, which is disclosed hereinbelow. A 
method clean 94 deletes table elements. A method getElemen- 
5 tAt 96 returns the value of an element in the table that is 
specified by an index parameter. A method isEmpty 98 determines 
if the table is empty. A method setElementAt 100 sets a value 
for an element of a specified table. 

[0057] The interface TableDataAccess 90 relates to a 

10 class Table 102, which is an abstract class representing a ge- 
neric table in the session database. A number of useful fields 
are defined in the class Table 102. A field datalD 104 identi- 
fies the data in the table. Whenever the table content is 
changed, a counter is incremented. Iterators maintain a snap- 

15 shot of the data identified by the field datalD 104 datalD for 
purposes of data validation. An example of such an iterator is 
provided by a method getRecords 106, which sequentially re- 
trieves all the records from the table. Another example is a 
method f ilterRecords 108, which sequentially retrieves all the 

20 records from the table, according to filtering criteria set by 
the user. 

[0058] Links to other classes that relate to or access 

the class Table 102 are present. For example, a field recordAd- 
der 110 holds a link to a class recordAdder, which is disclosed 

25 hereinbelow. In another example, a field TableDataAccess 112 
holds a link to an instance of the interface TableDataAc- 
cess 90. The methods of the class Table 102 include a method 
addRecord 114, which appends a new record to the end of the ta- 
ble. Another method used for record addition to a table is a 

30 method completeRecords 116, which adds only those records that 
are present in the platform and in a current session. A method 
computeKey 118 determines the key of a specified record. A 
method deleteRecord 12 0 functionally removes a record from the 
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table. However, instead of actual removal the current implemen- 
tation marks the record ID in order to add it to a list of de- 
leted records. Later, the above-noted iterators determine 
whether the next record is included in the list of deleted re- 
5 cords, and if so, such a record is not processed. As a result, 
the deleted record is not included in report documents, such as 
session or results report documents. The next time the session 
is loaded, such a deleted record is actually deleted. A method 
free 122 clears table data. A method getRecordsCount 124 counts 

10 all undeleted records in the table. A method getAllRecord- 
sCount 12 6, used by the iterators, counts all the records in 
the table, including deleted records. A method getKeywords 12 8 
returns a list of available keywords in the table. A method 
getRecord 13 0 returns a record from the table according to a 

15 record index. A method getRecordlndex 132 supplies a record in- 
dex in the table based on a unique key of a record. The method 
getRecordlndex 132 offers a rapid way to search the table, but 
must be used with caution, as it is not guaranteed that an in- 
dex continues to point to the same record after the table 

2 0 changes, for example after a record has been removed. A method 
searchRecord 134 returns a record from the table based on its 
unique key. A method updateRecord 13 6 updates a table record 
that is specified by its record index. 

Listener . 

25 [0059] An interface ListenerNotif ier 138 delegates 

listener notification functionality to certain session classes 
as detailed hereinbelow. A method notif yListeners 140 notifies 
listeners of a session event. Many of the classes disclosed 
with reference to Fig. 5 implement the interface ListenerNoti- 

30 fier 138. 

Test Suite Management. 

[0060] A class XMLAccessManager 142 is responsible for 

managing a queue of file operations that are requested by a 
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user, and has the following methods: A method startGroup 144 
collects components that may be needed to satisfy a request. 
For example, a list of included tests and excluded tests could 
be collected from the class Session 46. A method dispatch 146 
5 adds an action request to be executed to the queue. A method 
run 14 8 initiates execution of a thread corresponding to an ac- 
tion request at the head of the queue. 

[0061] An interface RunnableRequest 150 relates to the 

class XMLAccessManager 142. The interface RunnableRequest 150 

10 is implemented by each operation of the class XMLAccessMan- 
ager 142. It has a method cancel 152, which cancels a pending 
request. A method getRequestName 154 returns an identification 
of a pending request. A method getType 156 returns a category 
of a pending request. A method isFinished 158 returns a Boolean 

15 value that indicates whether a request has completed. 

[0062] A class ThreadsQueue 160 represents a queue of 

action requests executed as threads, and has the following 
methods. A method enqueue 162 inserts a new request into the 
queue. A method dequeue 164 returns the next pending request 

20 from the queue. A method getlterator 166 returns a record it- 
erator appropriate to a current action request. 

[0063] A class Def aultTestSuitesManager 168 is respon- 

sible for managing test suites, and has the following methods: 
A method setRecordAdder 170 sets a component responsible for 

25 adding properties to a test suite. A method setRestoreTSM 172 
sets a test suite that stores information before any changes 
are made to the test suites. When saving the current session 
and platform, test suites being saved are compared to this test 
suite to determine changes made during the current session. A 

3 0 method getFrameworkConf iguration 174 returns configuration in- 
formation for the test framework. A method readTestSuites 176 
reads the test suites from current platform and session files. 
A method getPlatf ormName 178 returns the name of the current 
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platform. A method setSessionParameters 180 sets configuration 
parameters for a current session. A method register- 
TestSuites 182 reads test suites from the platform file. The 
method registerTestSuites 182 is invoked during the creation of 
5 a new session. It reads test suite properties from storage and 
registers these with the new session. A method up- 
dateTestSuites 184 updates test suites in the current platform 
of an existing session according to information stored in test 
suite information files. A method setCurrentTestSuite 186 sets 

10 the test suite currently being executed. A method getCurrent- 
TestSuite 188 returns the test suite currently being executed. 
A method addListener 190 adds a specified test suites manager 
listener to receive action events from a test suites manager 
implementation object. A method removeListener 192 removes a 

15 specified test suites manager listener so that it no longer re- 
ceives test suites manager events from the current test suites 
manager. A method fireEvent 194 sends test suites manager 
events to the current test suites manager, which notifies its 
listeners in turn. A method setPlatf ormName 196 changes the 

20 name of the current platform, and associates the current plat- 
form name with all test suites. A method register-TestSuite 198 
adds a test suite to be handled by the current test suites man- 
ager. A method listOf TestSuiteNames 200 returns a list of test 
suite names being handled by the current test suites manager. A 

25 method not if yListeners 202 notifies listeners on an appropriate 
view event, for example a change in the state of a displayable 
object. A method removeTestSuite 204 removes a specified test 
suite. A method addTestSuite 206 adds a specified test suite to 
the list of test suites. A method save 208 saves the test 

30 suites to the current platform file. \ 
[0064] A class Def aultPlatf orm 210 relates to the 

class Def aultTestSuitesManager 168. It provides access to plat- 
form data of a current session. For example, MIDP and MIDP-for- 
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Palm are typical test platforms. Platform data includes test 
suites associated with the current test platform and persistent 
environment information. The class Def aultPlatf orm 210 has the 
following methods. A method isPlatf ormUpdated 212 compares ver- 
5 sion numbers of a default configuration and default platform, 
returning a Boolean value that indicates whether the current 
platform has been updated. A method getPlatf ormName 214 returns 
the name of the current platform. A method setPlatf ormName 216 
changes the name of the current platform. A method getView 218 

10 returns a requested view as specified by a view type. A method 
getTestSuitesManager 22 0 provides access to the test suites 
manager associated with the current platform. A method getPro- 
tectedPlatf orm 222 restricts access to the current platform. A 
method save 224 saves the platform associated with the current 

15 profile. A method getProf ileName 226 returns the name of the 
test suite record profile with which the current platform is 
associated. A method update 228 updates the content of the cur- 
rent platform. 

[0065] A class DeveloperTest SuitesManager 230 is re- 

20 sponsible for managing test suites, and relates to the class 
XMLAccessManager 142 and a class DeveloperSession 232. It has 
the following methods. A first method addListener 234 adds a 
session listener. A second method addListener 236 adds a speci- 
fied test suites manager listener to receive action events from 
25 a TestSuitesManager implementation object. A method 
addTestSuite 238 adds a test suite specified by name to a list 
of test suites. A method fireEvent 240 sends a test suites man- 
ager event to the current test suites manager, which notifies 
its listeners in turn. A method getFileName 244 is used in the 
3 0 current test suite to obtain the name of a XML file that it 
should write or read. A method getFrameworkConf iguration 246 
returns current configuration information about the test frame- 
work. A method getPlatf ormName 248 returns the name of the cur- 
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rent platform. A method getTestSuite 250 returns a test suite 
object specified by its name. A method getXMLAccessManager 252 
returns the current XML access manager as an instance of the 
class XMLAccessManager 142. A method listOf TestSuiteNames 254 
5 returns a list of test suite names being handled by the current 
instance of the class DeveloperTestSuitesManager 230. A method 
notify 255 notifies listeners on a test suites manager event. A 
method notif yListeners 256 notifies listeners on a session view 
event. A method registerTestSuite 258 adds a test suite that 

10 would be handled by the current instance of the class Develop- 
erTestSuitesManager 230. A method readTestSuites 260 reads test 
suites from the current platform file. A method removeLis- 
tener 262 removes a specified test suites manager listener, so 
that it no longer receives test suites manager events from the 

15 current instance of the class DeveloperTestSuitesManager 230, A 
method removeTestSuite 264 removes a specified test suite from 
the list of test suites being managed by the current instance 
of the class DeveloperTestSuitesManager 230. A method 
searchTestSuiteName 266 locates a specified test suite in a 

20 current list of test suites. A method setCurrentTestSuite 268 
designates a test suite to be executed. 

[0066] Fig. 5' illustrates further details of the 

above-noted class record 60. An instance of the class record 60 
represents a single record of a table. The class record 60 has 

25 a method getField 272, which returns a field specified by a 
field index. A method getFieldlndex 274 returns a field index 
corresponding to a field name. A method getFieldNames 276 re- 
turns the names of fields specified by one or more field indi- 
ces. A method getValue 278 returns the value of a test record 

30 field that is specified by a field index. A method setField 280 
sets the contents of a field specified by a field index. A 
method setValue 282 sets the value of a field specified by a 
field index. A method size 284 returns the size of a record. A 
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method toString 286 creates a string representation of a re- 
cord. 

[0067] A class TestRecordlmpl 288 relates to the class 

record 60 and holds information concerning tests in test . 
5 suites. It has a method clone 2 90, which returns a copy of a 
specified test record. 

[0068] A class ExcludedRecordlmpl 292 relates to the 

class record 60 and holds information concerning tests in ex- 
cluded test suites. It has a method clone 294, which returns a 
10 copy of a specified excluded record. 

[0069] A class TestSuiteRecordlmpl 296 relates to the 

class record 60 and is adapted for test suite records. It has a 
method clone 298, which returns a copy of a specified test 
suite record. A method getTSName 3 00 returns the name of the 
15 test suite to which a test suite record belongs. A method get- 
Value 3 02 returns the value of a test record field that is 
specified by an index value. A method setValue 3 04 sets the 
value of a test record field that is specified by an index 
value . 

20 [0070] A class Environment Recordlmpl 306 relates to the 

class record 6 0 and is adapted for environment records, which 
contain fields and field value retrieval methods. It has a 
method clone 308, which returns a copy of a specified environ- 
ment record. 

25 [0071] A class SelectedRecordlmpl 310 relates to the 

class record 60 and is adapted for selected tests, each of 
which represent the name of a test or a package that were se- 
lected in a session. It has a method clone 312, which returns a 
copy of a specified selected record. 

30 [0072] A class Def aultHTMLReport 314 generates a test 

execution report in HTML format. Its principal function is a 
method writeHTMLreport 316, which processes page generation in 
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HTML format. It returns the value true if a report has been 
successfully generated. 

[0073] A class Fil terlterator 318 relates to the class 

record 60. The class Filterl terator 318 enables iteration of 
5 records in a table by filtering, and is used as a wrapper of 
other record iterators. It has the following methods. A method 
currentRecordID 320 a session table identifier of a current re- 
cord, typically retrieved by the methods next or previous, 
which are disclosed in further detail hereinbelow. A method 

10 skipForward 322 skips a specified number of records in a for- 
ward iteration. A method skipBackward 324 skips a specified 
number of records in a backward iteration. A method hasPrevi- 
ous 326 returns the value true if a reverse iteration has more 
records. A method previous 328 returns the previous record in a 

15 record iteration. A method hasNext 330 returns the value true 
if a forward iteration has more records to be accessed. A 
method next 332 returns the next record in a record iteration. 
A method remove 334 removes a specified record. 

[0074] A class XMLAccess 336 is a utility class for 

2 0 reading and writing session and platform data in the well-known 

XML format. It includes the following methods. A method 
write 338 writes a specified test suite, or other records of a 
specified type to a XML file. A method notif yListeners 340 no- 
tifies listeners on an appropriate view event. A method 
25 read 342 reads file content of a specified type into a linked 
list of records. A method merge 344 merges a list of files into 
a single output file. 

[0075] A class EnvironmentBuilder 346 is a utility 

class, which configures the platform and session environment. 

3 0 The class EnvironmentBuilder 346 loads an environment configu- 

ration file into memory, and provides the following methods for 
extracting information regarding the environment. A method 
buildEnvironmentRecords 348 generates a linked list of records 
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containing environment information. A method getAllRecords 350 
returns all environment records as a linked list. A method get- 
Platf ormRecords 352 returns platform environment information as 
a linked list of records. A method getSessionRecords 354 re- 
5 turns session environment information as a linked list of re- 
cords . 

[0076] The class DeveloperSession 232 provides access 

to session data of the current session. The class DeveloperSes- 
sion 232 is built on the current platform, and is responsible 

10 for handling data that is specific to the current session, such 
as session environment modification, test suite environment 
modification, and test selection. It has the following methods: 
A method addListener 356 adds a listener to session events. A 
method getPlatform 358 returns the platform on which the cur- 

15 rent session relies. A method getPlatf ormName 360 returns the 
name of the current platform. A method getProtectedSession 362 
restricts access to the current session. A method getSession- 
Name 3 64 returns the name of the current session. A method 
getTestSuitesManager 366 provides access to the current plat- 

20 form test suites manager, for example an object of the class 
Def aultTestSuitesManager 168. A method getView 368 returns an 
appropriate view based on a specified view type. A method has- 
Conflicts 370 determines if there are conflicts in current ses- 
sion and platform properties. A method loadResults 372 loads 

25 results of the current session. A method mergeResults 374 
merges a list of results into one result file. A method noti- 
fyListeners 376 notifies listeners on an appropriate view 
event. A method save 378 saves the current platform. A method 
saveResults 3 80 saves results of the current session. A method 

30 setPlatf ormName 382 changes the name of the current platform. A 
method setSessionName 384 changes the name of the current ses- 
sion. A method update 386 updates content of the current plat- 
form. 
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[0077] A class SessionAccess 388 provides access to in- 

formation about the current session of the test framework, and 
relates to the class Def aul tTestSuitesManager 168, the method 
clone the class XMLAccessManager 142, and the class Develop- 
5 erSession 232. It has a method setCurrentSession 390, which 
specifies the current session by name. 

[0078] A class Platf ormAccess 392 provides access to 

information about the current platform, and relates to the 
class XMLAccessManager 142. It has a method setCurrentPlat- 

10 form 394, which sets the current platform according to a speci- 
fied name. A method addSessionAccess 396 returns an instance of 
the class SessionAccess 388 to the caller. 

[0079] A class Def aultSession 398 provides access to 

session data of a current session, and is responsible for han- 

15 dling data specific for the current session, such as environ- 
ment modification, test suite environment modification, and 
test selection. It has the following methods. A method addLis- 
tener 400 adds a listener on session events. A method getPlat- 
form 4 02 returns the platform on which the current session is 

20 built. A method getPlatf ormName 404 returns the name of the 
current platform. A method get Prof ileName 406 returns the name 
of the test suite record profile with which the current plat- 
form is associated. A method getProtectedSession 408 restricts 
access to the current session. A method getSessionName 410 re- 

25 turns the name of the current session. A method getTestSuites- 
Manager 412 provides access to the test suites manager of the 
current platform. A method getView 414 returns a view based on 
a specified view type. A method hasConflicts 416 returns the 
value true if there are conflicts in session and platform prop- 

30 erties. A method isPlatf ormUpdated 418 returns a Boolean value 
that indicates whether the current platform has been updated. A 
method loadResults 420 loads results of the current session. A 
method mergeResults 422 merges multiple result files into one 
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file. A method not if yListeners 424 notifies listeners on a 
specified view event. An overloaded identifier designates meth- 
ods save 426, 428, 430. The method save 42 6 saves the session, 
including its tests, environment and test suites. The method 
5 save 428 saves the current platform. The method save 430 saves 
test execution results according to their type, e.g., passed, 
failed. A method saveResults 432 saves results of the current 
session. A method setPlatf ormName 434 changes the name of the 
current platform. A method setSessionName 436 changes the name 

10 of the current session. A method update 438 updates content of 
the current platform. 

[0080] A class Def aultSessionManager 440 manages all 

session and platform related operations in the test framework, 
and has the following methods. A class DeveloperSessionMan- 

15 ager 442 is similar to the class Def aultSessionManager 440, but 
is adapted for use by commercial software developers. Both 
classes have the following methods. A method createPlatf orm 444 
creates new platform. A method createSession 446 creates a new 
session based on the current platform. A method getAvailable- 

20 Platf ormsNames 448 searches for existing platforms. A method 
getAvailableSessionNames 450 searches for existing sessions for 
the current platform. A method getTestSuitesManager 452 returns 
a test suites manager object for those test suites that are as- 
sociated with the current platform. A method getUIComponent 454 

25 returns user interface implementation of the test framework. A 
method loadResults 456 loads previous tests result from a file. 
A method mergeResults 458 merges a list of results into one re- 
sult file. A method openPlatform 460 opens an existing plat- 
form. A method openSession 4 62 opens an existing session. An 

30 overloaded identifier designates methods savePlatform 464, 
466, both of which save the current platform. A method saveRe- 
sults 468 saves results of test execution, including current 
configuration information, e.g., environment, and test configu- 
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ration information. An overloaded identifier designates a 
method saveSession 470, which saves the current session. 

[0081] A class DeveloperPlatf orm 472 is similar to the 

class Def aultPlatf orm 210, but is adapted for commercial soft- 
5 ware development. It has the method getPlatf ormName 214, the 
method setPlatf ormName 216, the method getView 218, the method 
getProtectedPlatf orm 222, and the method update 228. An over- 
loaded identifier designates the method save 224 and another 
method save 474. The method save 474 saves the platform con- 
10 figuration by saving each view. The class DeveloperPlatf orm 472 
also has a method getTestSuitesManager 476, which provides ac- 
cess to the platform test suites manager. 

Extensions and Subsidiary Classes. 

[0082] The class Table 102 relates to a class Compari- 

15 sonTable 478, which reads two session results and creates a ta- 
ble, which is a comparison of the two. 

[0083] A class Def aultResultsComparator 480 relates to 

the class ComparisonTable 478, A method getComparisonRe- 
cords 4 82, returns a list of all comparison records, or a sub- 

2 0 set of comparison records according to a prefix pattern. 

[0084] The class Table 102 relates to a class Select- 

edTestsTable 484, which holds the tests that are selected in a 
current session. The class SelectedTestsTable 484 has the fol- 
lowing methods: A method computeKey 486 computes a key for a 
25 test record package, or computes an equivalent TestRecord key 
based on a package, class and test case name. A method cre- 
ateRecord 488 creates a specified record. 

[0085] The class Table 102 relates to a class Exclud- 

edTestsTable 490, which holds the tests that are excluded in a 

3 0 current session. The class ExcludedTestsTable 4 90 has the meth- 

ods 48 6, 4 88. 

[0086] A class TestsTable 492 extends the class Ta- 

ble 102, and has the following methods: A method addRecord 4 94 
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overrides the method addRecord of the superclass, that is the 
class Table 102, in order to generate counting events, which 
may result in a counter being incremented in another class. The 
class TestsTable 492 has the methods 486, 488. 
5 [0087] The class Table 102 relates to a class Protect- 

edView 496, which protects write access to the current plat- 
form. The class ProtectedView 496 has the following methods: A 
method addListener 4 98 adds a session listener. A method coun- 
tRecords 500 counts the number of records in a class view 502, 

10 which is disclosed hereinbelow. A method getRecord 504 returns 
a view record that is identified by its unique ID. A method 
getRecords 506 returns all view records according to filters 
specified in a session query. If a session query is not speci- 
fied, or the value null is supplied, then all records are re- 

15 trieved by the method getRecords 506. A method getViewOb- 
ject 508 returns an object type of the class view 502. A method 
initialize 510 initializes a linked list of records. A method 
notif yListeners 512 notifies listeners on session events. A 
method removeListener 514 removes a session listener. A method 

20 removeListeners 516 removes all registered listeners. A method 
setRestoreView 518 restores an environment view. The class rep- 
resentation of an environment view is disclosed hereinbelow. 

[0088] The class Table 102 relates to the above- 

mentioned class view 502, which provides basic access to the 

25 session table records. It is extended by other views, as dis- 
closed herein, and has the following methods. A method addLis- 
tener 520 adds a session listener. A method get Frame workCon- 
figuration returns the configuration of the test framework. The 
class view 502 has the method countRecords 500, the method 

30 getRecords 506, the method removeListeners 516, and the method 
notif yListeners 512. A method getTable 522 returns a reference 
to a table of a subclass. A method getView 524 creates a vir- 
tual view, which is a subset of a View based on a session 
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query. If no session query is specified or the value null is 
supplied, a reference to the current view is returned. A method 
initialize 526 initializes the environment table of the current 
session, and in a variant, initializes the appropriate table of 
5 the current session. A method initializeTable 528 registers a 
request for initializing a table based on a linked list. A 
method read 53 0 processes a request to read from a view. 

[0089] A class EnvironmentTable 532 extends the class 

Table 102, and is used to provide view of records of known 

10 platform environments. It has the method computeKey 486, the 
method createRecord 488, the method free 122, and the method 
searchRecord 134 Additionally, a method getEnvironmentRe- 
cords 534 returns an environment record from a table. A method 
getExecutionMode 536 returns a current execution mode. 

15 [0090] A class TestSuitelmpl 538 extends the class Ta- 

ble 102, and. The class TestSuitelmpl 538 encapsulates informa- 
tion about a particular test suite, e.g., its properties and 
name. It has the method computeKey 486, the method createRe- 
cord 488 and a method addRecord 540. Additionally, a method se- 

2 0 tRecordAdder 542 sets a component responsible for adding prop- 

erties to a test suite, and overrides a method setRecordAdder 
in the class Table 102 . A method setRestoreTestSuite 544 sets 
another instance of a test suite for the purpose of properties 
comparison when saving the test suite later. A method getCate- 
25 gories 546 returns a list of categories of the current test 
suite. A method getTestSuiteRecords 548 returns records that 
belong to a specified category. If. the value null is passed to 
the method getCategories 546, then only those records that be- 
long only to the current test suite are returned. A method 

3 0 getRecordsByScope 550 returns records according to a specified 

scope, for example, whether the record is available in the 
platform editor 28 (Fig. 1), or at the client. A method 
testSuiteUpdated 552 updates a specified test suite according 
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to values passed to the method. A method getTestSuiteProper- 
ties 554 returns the properties of a test suite as a properties 
object. A method getTestSuiteProperty 556 returns a requested 
test suite property according to a specified test suite key. A 
5 method save 558 saves the current test suite as a XML file. 

[0091] A class RecordAdder 560 relates to the class Ta- 

ble 102, and has methods adapted to the addition of records to 
a table. The class RecordAdder 560 determines if a record was 
administratively modified by the platform editor 28 (Fig. 1) 

10 and also modified by the user at the test console, or if it was 
modified only in one of them. If the record was modified in 
both components, then it is added to a table AskTable, which is 
described hereinbelow. The table AskTable is later used in or- 
der to ask the user which modified value to accept. Otherwise, 

15 if the record was modified only in the platform editor 28 
(Fig. 1) , then the administrative value is accepted automati- 
cally. If the record was modified only by the user, then the 
user's modification is accepted. Some methods of the class Re- 
cordAdder 560 return an object of type AskTable, as described 

20 with reference to a class AskTable 562. 

[0092] The class AskTable 562 is used to manage a table 

for representing conflict data in the session and platform. It 
inherits methods from a standard Java class j ava . awt . Container , 
and two J2SE Java classes, j avax. swing . Jt able and 

25 j avax. swing . Jcomponent , available from Sun Microsystems, Inc. 
Additionally, the class AskTable 562 has a method getName 564, 
which returns the name of a current table. A method updateRe- 
cords 566 updates records according to the outcome of conflict 

resolution and record merge activity. A method add 568 adds a ( 

i 

30 record to a current table. A method getNumOfRows 570 returns .i 
the number of rows in a current table. 

[0093] The interface TableDataAccess 90 relates to a 

class DirectDataAccess 572, which offers direct random and se- 
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quential access to a table. The class DirectDataAccess 572 in- 
herits the methods 92 , 94 , 96, 98, 100 from the interface Ta- 
bleDataAccess 90. 

[0094] The interface TableDataAccess 90 relates to a 
5 class PagedDataAccess 574, which offers paged random and se- 
quential access to a table. The class PagedDataAccess 574 in- 
herits the methods 92, 94, 96, 98, 100 from the interface Ta- 
bleDataAccess 90. 

Views • 

10 [0095] A class EnvironmentViewImpl 576 provides a view 

of environment records in a session, and has the following 
methods. A class DeveloperEnvironmentViewImpl 578 is similar to 
the class EnvironmentViewImpl 576, but is adapted to use by 
commercial software developers. Both classes have the following 

15 methods. A method getCategories 580 returns a list of catego- 
ries in the current test suite. A method getEnvironment Re- 
cord 582 returns an environment record that is specified by 
name. A method getEnvironmentRecords 584 returns an environment 
record from a table as specified by a record index. A method 

20 getPlatf ormName 586 returns the name of the current platform. A 
method getSessionRecords 588 returns session environment infor- 
mation as a linked list of records. A method getTable 590 re- 
turns a requested table. A method save 592 saves the content of 
the table associated with a current view. A method setEnviron- 

2 5 mentRecord 5 94 saves an environment record. 

[0096] A class ProtectedEnvironmentView 596 protects 

write access to an environment view, and has the method get- 
Categories 580, the method getEnvironmentRecord 582, and the 
method getEnvironmentRecords 584. A method getView 608 creates 

3 0 a virtual view, which is a subset of the current view based on 

a session query. If the query is not specified or if the value 
null is supplied, a reference to the current view is returned. 
A method initialize 610 initializes a specified table of the 
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current session. A method save 612 saves the current view as a 
XML file. 

[0097] A class TestsViewImpl 614 represents a session 

view of test records, and has the following methods. A method 
5 availableExecutionStatuses 616 returns a list of execution 
statuses in the current session. Generally, these are obtained 
from an immediately prior test execution. If a subset of tests 
is being rerun, then the list of statuses remains unchanged 
from the prior test suite execution. A method availableKey- 

10 words 618 returns a list of keywords available in the current 
session. Keywords are strings assigned to each test according 
to its type. For example, an interactive test has an "interac- 
tive" keyword. A method getTable 62 0 returns a requested table. 
A method getTestRecord 622 returns a test record that is speci- 

15 fied by its package, class and case name. A method getTests- 
Viewlnfo 624 returns test view information. A method initial- 
ize 626 initializes a table of the current session. A method 
prepareForTestExexcution 628 enables the current session to 
perform necessary initialization prior to test execution. A 

2 0 method save 63 0 saves content of a table of the current view. A 
method saveTestFailure 632 saves a test failure report for a 
test that is specified uniquely by name, class and package. A 
method saveTestResult 634 saves a test result report, which is 
further specified by test name, class and package in order to 

25 provide a unique identification of the test result in binary 
form. A method setFileName 63 6 sets the name of the session for 
the current view. A method setPlatf ormName 63 8 changes the name 
of the current platform. A method setSelectedTests 640 is an 
iterator of selected tests to be saved directly as a XML file. 

30 [0098] A class ProtectedTestsView 642 protects write 

access to a test view, and has the method getView 608, the 
method initialize 610, the method save 612, the method getTes- 
tRecord 622, the method prepareForTestExecution 62 8, the method 
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method saveTestFailure 632, the method saveTestResult 634, the 
method setFileName 636, the method setSelectedTests 640, and 
also has a method getTestsViewInf o 644, which returns requested 
tests view information . 
5 [0099] A class ExcludedTestsViewImpl 646 provides a 

session view of excluded test records, and has the following 
methods. A method generateExcludeRecord 648 returns a record of 
a test that was selected to be excluded from a test suite. A 
method get InitExcludeList 650 returns a linked list of tests 

10 that are excluded from performance in a current test suite. A 
method getRecords 652 returns all view records using filters 
specified in a session query. A method getTable 654 returns a 
reference to a table of a subclass. A method isExcluded 656 
tests for the existence of a test having a specified name in 

15 the excluded test list of the current session. A method perfor- 
mExcludeTransaction 658 adds excluded tests to the excluded 
test list, and removes tests that were previously excluded but 
are no longer excluded from the list. The class ExcludedTests- 
ViewImpl 64 6 also has the method setFileName 63 6 and the method 

20 setPlatf ormName 638. 

[0100] A class SelectedViewImpl 660 provides a session 

view for the test records, which are not excluded from perform- 
ance. The class SelectedViewImpl 660 has the method getRe- 
cords a805, the method getTable 654, the method setFile- 

25 Name 636. A method generateSelectedRecord 662 returns a record 
of a specified test. A method isSelected 664 tests for exis- 
tence of a specific name in a selected test list of the current 
session. A method perf ormSelectTransaction 666 adds selected 
tests to the selected test list, and removes tests that were + 

30 previously selected but are no longer selected from the list. 

Recordlterator . 

[0101] The class Table 102 relates to an interface Re- 

cordlterator 668, which has several implementing classes. It 
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includes methods, which are replicated in the classes. For ex- 
ample, a method skipForward 670 and a method skipBackward 672 
skip records in forward and backward directions, respectively. 

[0102] A class Baselterator 674 enables iteration of 

5 records, even if the vector, which is the list of records, is 
under current modification) . Methods of the class Baseltera- 
tor 674 include a method currentRecordID 676, which returns the 
position of a current record in the table. 

[0103] A class Reverselterator 678 enables reverse it- 

10 eration. This is done through caching iterated record ID'S. The 
records can also be iterated in a forward direction as well, 
until the all the cached ID'S limit have been processed. The 
class Reverselterator 678 has the method currentRecordID 676, 
the method skipForward 670 and the method skipBackward 672. 

15 [0104] A class Exact Iterator 680 returns records ac- 

cording to their ID'S. The class Exactlterator 680 has the 
method currentRecordID 676, the method skipForward 670 and the 
method skipBackward 6 72 . 

[0105] A class Viewlterator 682 returns specified 

2 0 views, and has the following methods. A method add 684 adds a 
record to a current list of records to be viewed. A method com- 
pleted 686 signals completion of view generation in the itera- 
tion. A method hasNext 688 returns the value true if a forward 
iteration has more elements. A method next 690 returns the next 

25 element in a view iteration, A method start 692 signals the 
start of test record collection into a list. 

Miscellaneous Classes and Interfaces. 

[0106] A class MonitorDialog 694 indicates progress of 

saving or opening a session by displaying a progress bar in a 
30 modal dialog box. It is a slow method, and is therefore run in 
a separate thread. It has a method setDescription 696, which 
returns a descriptor of the current session. Its principal 



P9307 



47900 



34 



Ver. 47900S9.doc 



method is a method run 698, which displays the dialog box and 
progress bar. 

[0107] A class Recordlnf olmpld 700 provides access to 

specified records, and has the following methods. A method 
5 getRecord 702 returns a specified record object. A method 
getRecordID 704 returns a record Identifier in the current 
view. A method isLastTest 706 returns a Boolean value, accord- 
ing to whether the current record is the last of the records 
selected for test execution. 

10 [0108] A class TableDataAccessFactory 708 creates ob- 

jects of the interface TableDataAccess 90, and has the follow- 
ing methods. An overloaded identifier designates a method cre- 
ate 710 and a method create 712, both of which return objects 
of type TableDataAccess. 

15 [0109] A class TestSuitelnstaller 714 is responsible 

for the actual installation of a test suite, and has the fol- 
lowing methods. A method installTestSuite 716 is the principal 
method of the class, which installs a designated test suite, 
and returns an object of type class TestSuitelnstaller 714. A 

20 method addListener 718 adds a listener on the event of a test 
suite installation. A method notifyThis 720 notifies listeners 
that a test suite has been successfully installed. 

[0110] A class ProtectedPlat f orm 722 is responsible for 

protecting write access to a designated platform, and has the 

25 following methods. A method getPlatf ormName 724 returns the 
name of the current platform. A method getProf ileName 726 re- 
turns the name of the test suite record profile with which the 
current platform is associated. A method getTestSuitesMan- 
ager 728 provides an access to the test suites manager associ- 

30 ated with the current platform. A method GenericView 730 re- 
turns the view associated with a specified view type. A method 
isPlatf ormUpdated 732 compares version numbers of a default 
configuration and default platform, returning a Boolean value 
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that indicates whether the current platform has been modified. 
An overloaded identifier designates a method save 734 and a 
method save 736, both of which save the current platform. A 

1 method setPlat f ormName 738 changes the name of the current 
5 platform. 

[0111] A class Conf lictWizard 740 is responsible for 

dealing with value conflicts in the current session and plat- 
form. Its principal method is a class constructor ConflictWiz- 
ard 742, which implements a dialog that is shown to the user 

10 when the test framework loads in order to resolve values, which 
conflict in the session and platform. A method notif yListen- 
ers 744 notifies listeners on the appropriate view event. 

[0112] A class TestResultsDecoder 746 has functions for 

presenting information concerning executed tests. A method 

15 get Test Comments 748 returns messages generated by the test 
framework during execution of a current test. A method getTest- 
Log 750 returns contents of a log file generated by the test 
framework during execution of the current test. A method 
getTestStatus 752 returns status information concerning the 

2 0 current test. 

[0113] Requests for test execution are handled via an 

interface Request 754. It has a method perform 756, which actu- 
ates the request in the test framework when an appropriate ob- 
ject is passed as a parameter to the method. 
25 [0114] A class ProtectedSession 758 protects write ac- 

cess to a session, and has the following methods. A method ad- 
dListener 760 adds a listener on session events. A method get- 
Platform 762 returns the platform on which the current session 
is constructed. A method getPlatf ormName 764 returns the name 

3 0 of the current platform. A method get Prof ileName 766 returns 

the name of the test suite record profile with which the cur- 
rent platform is associated. A method getProtectedSession 768 
returns a reference to a current session having restricted ac- 

P9307 



47900 



Ver. 47900S9.doc 



36 

cess. A method getSessionName 770 returns the name of the cur- 
rent session. A method getTestSuitesManager 772 provides access 
to the test suites manager associated with the current plat- 
form. A method getView 774 returns a requested view as speci- 
5 fied by a view type. A method hasConflicts 776 determines if 
there are conflicts in current session and platform properties. 
A method isPlat f ormUpdated 778 returns a Boolean value that in- 
dicates whether the current platform has been updated. A method 
loadResults 780 results of the current session. A method 

10 mergeResults 782 merges a list of results into one result file. 
An overloaded identifier designates three methods save 784, 
786, 788. The methods save 784, 786 save the current platform. 
The method save 788 saves a session specified by type. A method 
saveResults 790 saves results of a specified session. A method 

15 setPlatf ormName 792 changes the name of the current platform. A 
method setSessionName 794 changes the name of the current ses- 
sion. A method update 7 96 updates content of the current plat- 
form. 

[0115] A class ServiceThread 798 asynchronously removes 

2 0 unused tables, and has the following methods. A method run 800 

overrides a method of the same name in the standard Java class 
java. lang. Thread from which it inherits, and initiates a new 
thread. A method removeDirectory 802 removes a table by delet- 
ing a subdirectory in which it is found. These methods are op- 
25 erative when access to the appropriate object is not re- 
stricted . 

[0116] A class HTMLTags 804 includes HTML tags needed 

to create a HTML report. In the current embodiment, tags are 
generated as strings taken from the standard Java class 

3 0 j ava . lang . String . For example, A method newLine 806 generates a 

string newLine, which is the string token pair "<br>/br>/". 

[0117] It will be appreciated by persons skilled in the 

art that the present invention is not limited to what has been 
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particularly shown and described hereinabove. Rather, the scope 
of the present invention includes both combinations and sub- 
combinations of the various features described hereinabove, as 
well as variations and modifications thereof that are not in 
5 the prior art, which would occur to persons skilled in the art 
upon reading the foregoing description. 
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COMPUTER PROGRAM LISTINGS 

Listing 1 

class Record { 
5 /** The field for a property name */ 
public final int NAME = 0; 

/** «phe field for a property value */ 
public final int VALUE = 1 ; 

10 

/** 

* Scope of this property: 

* basic or advanced 
*/ 

15 public final int SCOPE = 2; 
/** 

* Profile to which this record belongs 

* platform or session 
20 */ 

public final int PROFILE = 3; 
/** 

* Logic category of this property 

2 5 **/ 

public final int CATEGORY = 4; 
/** 

* File type of this property 

3 0**/ 

public final int TYPE = 5; 

/** 

* execution Mode to which this record belongs 
35 */ 

public final int MODE = 6; 
/** 

* Execution SubMode to which this record belongs 

4 0 */ 

public final int SUBMODE = 7; 
/** 

* Documentation available only if this record 
45 * represents category or mode 

* * / 

public final int DOCUMENTATION = 8; 
/** 

50 * Description about sub modes available only if this 

* record represents mode 
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**/ 

public final int DESCRIPTION = 9; 
public static final int COUNTER=10; 
public static final int MODIFIED=1 1 ; 
5 } 

Listing 2 

Platform platform = 
10 Platf ormSessionManager . createAndSavePlat f orm( name ) 
// administrator of the test system 
// creates a new platform 

In the platform: 

15 TestSuitesManager . WriteRequest (test suites list of Record) 
//taken from the test suites directory 

EnvironmentView . WriteRequest (parameterslist of Record) 

20 SelectedView . WriteRequest ( selected tests list of Record) 
//can be selected by the platform editor 
//for each request: 

XMLAccess . write ( Record [ ] records ) 

25 



Listing 3 

30 // returns value of the parameter to 
// the other components*/ 

public String getValue ( int fieldlndex) { 

if (fieldlndex < 0 |j fieldlndex >= recordSize) 

{ 

3 5 System . out . print In ( " Invalid index" + fieldlndex ) ; 

return null; 

} 

else{ 

String value = values [ fieldlndex ] ; 
40 if (value ! =null && 

value . indexOf ( " ( PARENT_DIR) ")!=-!) { 
//if this parameter should be replaced 

value = session, getFrameworkProperty ( "parentDir" ) + 
45 File . separator + 

value . substring ( 1 3 ) ; 



50 



//replace the variable PARENT_DIR with it's value //that 
session took from local . configuration file 

value = replaceSeparators (value ) ; 
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//convert it's file separator according to the 
/ / operating system 
return value; 

} 

5 return value; 

} 
} 

/**Converts separators according to operating system 
10 **/ 

private String replaceSeparators ( String value) { 
String sep = File . separator ; 

15 if (sep.charAt(O) == '/') 

value = value . replace (' \\ ' , sep. charAt (0) ) ; 
else 

value = value . replace ('/' / sep . charAt (0) ) ; 
return value; 
20 } 

/** return replaced value to what it was before the 
replacement*/ 

25 public boolean setValue(int fieldlndex, 
String fieldValue) { 

if (fieldlndex < 0 || fieldlndex >= recordSize) 
return false; 

30 

String pDir = 
session . getFrameworkProperty ( "parentDir " ) ; 

if ( fieldValue !=null && fieldValue . startsWith ( { 
35 String value = " ( PARENT_DIR) " + 

fieldValue . substring ( fieldValue . indexOf (pDir ) + 
pDir . length ( ) ) ; 

value = replaceSeparators (value) ; 
fieldValue = value; 

40 } 

values [ fieldlndex ] = fieldValue; 

return true; 

} 

45 

Listing 4 

Platform platform = 
Platf ormSessionManager . openPlatf orm( valueOf (parentDir) 
+ name) 

50 //administrator of the test system creates a new //platform 
In the platform: 
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TestSuitesManager . ReadRequest ( test suites 

list of Record) 
//taken from the test suites directory 

5 EnvironmentView . ReadRequest ( parameterslist of Record) 

SelectedView . ReadRequest ( selected tests list of Record) 
//can be selected by the platform editor 

10 for each request : 

Recordf ] records = XMLAcce s s . read ( pi a t f o rmName ) 

Listing 5 

15 Platform , open ( f ileName ) -> 

EnvironmentView/TestSuitesManager . init ( ) ; 
BackupEnviornmentView/BackupTestSuitesManaget . init ( ) -> 

Platform . save ( f ileName ) ; 
TestSuitesManager . save ( BackupTestSuitesManager ) ; 



20 



Listing 6 



public void save (Record[ ] records) 
25 { 

try 
{ 

updateRecords ( thisTable , backupTable ) ; 
xmlAccessManager . dispatch ( new 
30 WriteRequest ( records , f ileName ) ) ; 

} 

catch ( Exception e ) 
{ 

35 e . getMessage ( ) ; 

e . print St ackTr ace ( ) ; 

} 

} 

40 private void updateRecords ( Table thisTable , Table 
restoreTable ) { 

Hashtable keys = thisTable . getKeys () ; 
Enumeration enum = keys . keys () ; 

45 while (enum . hasMoreElements ( ) ) { 

//for all the reords 
String key = ( String) enum . nextElement () ; 
Record oldRecord = restoreTable . searchRecord ( key ) ; 

50 //take record before the changes 

Record newRecord = thisTable . searchRecord ( key ) ; 
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10 



25 



35 



//take record after the changes 
if (oldRecord ! =null && newRecord ! =null ) { 
String oldValue = 

oldRecord . getValue ( EnvironmentRecord . VALUE ) ; 

//take the values 
String newValue = 
newRecord . getValue ( EnvironmentRecord . VALUE ) 



if ( oldValue ! =null && newValue ! =null ) { 
if ( ! newValue . equals ( oldValue ) ) { 

//compare the values - if different update 
// the counters 
15 newRecord . setValue ( newRecord . getFieldlndex 

("COUNTER") , 
String . valueOf ( Integer . parselnt 
(newRecord . getValue (EnvironmentRecord . COUNTER 
) )+1 ) ) ; 

20 newRecord . setValue ( newRecord . getFieldlndex 

("MODIFIED") , "true") ; 
} 



Listing 7 



Platf ormSessionManager . open ( platform) 
30 //loads platform records 

Platf ormSessionManager . open (session ) 

EnvironmentView/TestSuitesManager . init ( ) ; 
BackupEnviornmentView/BackupTestSuitesManaget . init ( ) 



//create backup holders of the Records 
Table . init ( SessionRecords , Platf ormsRecords ) 



//initialize Table with the records taken from the 
40 //Platform and from the Session 

for each Record 

Table . addRecord (SessionRecord, Platf ormRecord) 
RecordAdder . add ( SessionRecord , Platf ormRecord) 
45 //class that takes care about the / /updating/resolving 



Listing 8 



50 class RecordAdder 
{ 
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void addRecord( Record sessionRecord , Record 
platf ormRecord) { 

String sessionCounter = 
5 sessionRecord. getValue ( Record . COUNTER) ; 

int sCounter = Integer . parselnt ( sessionCounter ) ; 
//take Session's Record Counter 

10 String platf ormCounter = 

platf ormRecord . getValue ( Record . COUNTER) ; 

int pCounter = Integer . parselnt ( plat f ormCounter ) ; 
//take Platform's Record Counter 

15 

String sessionModif ied = 

sessionRecord . getValue ( Test SuiteRecord . MODIFIED) ; 
//true if Session Record was modified 

20 if (pCounter > sCounter) 

//if Platform's counter > Session's counter 
//then we should take value from the Platform, which 
// is the most updated 

autoReplace ( sessionRecord, platf ormRecord) ; 

25 else if (pCounter==sCounter && 

sessionModif ied . equalsIgnoreCase ( "true" ) ) { 
//if they are equal - then this is a conflict, 
// which the user should resolve 
AskUser ( sessionRecord, platf ormRecord, key) ; 

30 } 

else 

//Session is more updated - leave the value as is 
table . addRecord ( sessionRecord) ; 

} 

35 

private void autoReplace ( Record sessionRecord, 
Record platf ormRecord ) 

{ 

Record newRecord = new Record(); 
4 0 newRecord. setValue ( Record . MODIFIED , "true" ) ; 
} 

//clone values 

for (int i = 0; i < newRecord . size () ; i++) 
45 { 

newRecord . setValue ( i , platf ormRecord . getValue ( i ) ) ; 

} 

//after the replacement add to the Table 
50 table . addRecord ( newRecord) ; 
} 
} 
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